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The invention relates to a data processing device for suggesting a user, 
comprising a detector arranged for detecting a condition pertaining to the user of said device, 
and presentation means for presenting to the user a suggestion to perform an activity in 
response to detection of said condition. 

The invention also relates to a method of suggesting a user, comprising a step 
of detecting a condition pertaining to the user and a step of presenting to the user a 
suggestion to perform an activity in response to detection of said condition. 



a 

a 



Document WO90/1 5502 discloses a system comprising a timer and display 
means to display messages at predetermined intervals. Reminder messages such as reminding 
a viewer to run an errand, reminding an older viewer to take some medicine, or reniinding 
viewer of an appointment are extracted from a memory. The memory stores a start time and 
count time of the message, the message to be displayed upon a television screen. The start 
time and count time of the message may also be manually set by the user. The message may 
be created at a computer keyboard and transferred to the memory. 

The known system shows messages which are inputted and predetermined by 
the user. The user has to input text of the message or other information to be reminded about. 
Such system is rather passive and requires some of user's time which the user often does not 
have. The known system can display only the information inputted by the user, which is in 
most cases terse. The displayed text message may be not very direct and descriptive for the 
user. This passiveness of Ihe known system makes it unattractive for using. 



It is an object of the invention to provide a device of the kind defined in the 
opening paragraph which, when used by the user, actively assists the user in conducting his 
activities. 

This object is realized in that the device further comprises search means 
arranged to automatically find media data which provide the user with a direction for 
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perfonning said suggested user's activity, wherein said presentation means is arranged for 
presenting said found media data to the user. 

Hie detector may be arranged for detecting different conditions and situations 
such as a scheduled event in a user's agenda, a current user's activity, e.g. the user watches 
the TV, listens to the radio, reads a newspaper or hook, has breakfast etc. Upon detecting one 
or more of said conditions, the device provides to the user the suggestion to perform the 

snggActftd activity For exam ple, the device may suggest th e us er to change the current 

activity of watching TV to another suggested activity of doing physical exercise because the 
user has been sitting too long at the TV, the device may suggest to start preparing a meal one 
hour before a visit of friends, or the device may remind the user about some event. Such 
suggestions may be generated automatically. The search means is arranged for finding and 
retrieving the media data. The media data are preferably retrieved automatically by the search 
means so that the user does not necessarily have to input or select or take part in other way in 
finding and/or retrieving the media data. The device determines what media data can help the 
user in following the suggestion, e.g. in performing the suggested physical exercises, in 
preparing the suggested meal. Thus, when said media data are presented to the user, he/she is 
provided with an extensive description or example how to perform said activity. A video clip 
about cooking some meal, a video lesson of physical training, a video clip showing a clock 
from 5 minutes before some event till 1 minute are examples of such media data. The device 
according to the present invention actively provides to the user media data, which are usually 
information well perceivable by the user. That media data may also answer possible user's 
questions about the suggestion. 

The object of the invention also realized by the method of the present 
invention further comprising a step of automatically finding media data which provide the 
user with a direction for performing said suggested user's activity, and a step of presenting 
said found media data to the user. 



These and other aspects of the invention will be further elucidated and 
described with reference to the accompanying drawings, wherein: 

Fig. 1 shows a functional block diagram of the data processing device suitable 
for implementing the present invention; 

Fig. 2 shows an embodiment of the method according to the present invention. 
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Throughout the figures, same reference numerals indicate identical or 
corresponding components. 



With reference to Fig. 1, a functional block diagram of the data processing 
device 100 for suggesting a user 101 according to the present invention is shown. The device 
comprises a detector 110 arranged for detecting a condition 115 pertaining to the user 101. 
Such condition maybe, for example, a time or date of a user's personal schedule, a user's 
behavior or other. The device may optionally comprise a storage means (not shown) for 
storing data of the user's schedule in the device, and a timer (not shown) for counting the 
time in the device, wherein said timer may be coupled to the storage means and/or detector 
1 10. The detector may be arranged for monitoring the user's behavior to detect the condition. 
For instance, the detector may include a camera for taking pictures of Ihe user's environment 
and a recognition unit arranged for recognizing objects of the environment from video data 
recorded by the camera The detector may also include sound input means for inputting 
sound from the user's environment, wherein the sound data may be inputted to the 
recognition unit which may be arranged for recognizing a user's mood, a type of music 
reproduced in the environment etc. 

The conditions may be predetermined in the device and stored in the storage 
means, or the device may optionally comprise an analyzing unit arranged for establishing me 
condition, e.g., by analyzing repetitive events detected by the detector 1 1 0, or by adjusting 
the predetermined conditions to the individual user, or the like. 

The device is arranged for obtaining a suggestion 1 3 1 for the user to perform 
an activity, and the device comprises presentation means 120 for presenting said suggestion 
to the user 1 01 . The suggestions may be predetermined or generated in the device. 
The predetermined suggestions may be suggestions which are associated with corresponding 
predetermined conditions. The predetermined conditions to be detected and corresponding 
suggestions may be stored in the storage means. In one example, the user may schedule the 
device to remind the user at the predetermined time and/or date about some information, thus 
that time and/or date are predetermined conditions, that reminder information is the 
predetermined suggestion, as it is known in the prior art. 

The device may also comprise suggestion means 130 arranged for 
autonomously generating the suggestion 1 3 1 to perform the activity in response to detection 
of the condition. The word "autonomously means that the suggestion means can generate 
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the suggestions independently from the user. For example, the suggestion means may suggest 
the user to follow a diet and generates diet-suggestions because, for example, the user spends 
a lot of time passively without much of the physical activity and has a weight higher than in 
average, though the user has never been given the diet-suggestions. 
5 The prior art systems provide suggestions which are typically short and 

abstract for the user, e.g. a reminder "to prepare for the meeting", or a suggestion "to start 

d omg homework". Such suggestio n does not have sufficient semantic information for details 

which the user may need to bear in mind with that suggestion. For instance, from a 
suggestion "to start doing physical exercises" the user may not know or may not remember 
10 what exercises to do and/or and how exactly or properly to do them. To this end, the device 
according to the present invention actively participates in the process of suggesting the user 
by rendering to the user media data 141 which provide the user with a direction for 
performing the suggested user's activity 131. For example, a TV program, in which the 
physical exercises guiding the user to perform the suggestion are demonstrated, may be very 
15 useful for the user who is suggested "to start doing physical exercises". The user may simply 
and easily repeat the movements shown at that TV program, which is much better for the user 
than to be only suggested with the terse suggestion, e.g. in the form of text "do exercises X". 
Many other benefits of the present invention may be envisaged, e.g. the user may find that 
the shown physical exercises are new for him/her, the user may be simply pleased with 
20 surprising information, the user may learn that some suggested activity can be done in an 
alternative way or better, and so on. 

The device comprises search means 140 arranged to automatically find said 
media data 141 upon obtaining the suggestion 131. The presentation means 120 is arranged 
for presenting the found media data to the user 101. The media data may be audio data and/or 
25 video data and/or text data. The presentation means may comprise a speaker (not shown) to 
render audio media data and/or display (not shown) to render video media data and/or text 
data. Rendering the media data may also comprise changing parameters of the user's 
environment, such as an ambient light. Many implementations of the search means are 
possible. For example, the search means may be arranged to at first generate a request for the 
30 media data corresponding to a particular suggestion. Such request may be keywords used to 
find the media data which can elucidate the suggestion. The request may comprise not only 
information about the suggestion but also about the detected condition, and generally all 
information which may be potentially necessary for obtaining the media data upon obtaining 
the particular suggestion. For instance, the request may comprise information that the user 
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has to perform physical exercises of a moderate intensity for 10 minutes, and about the 
current time/date, and/or about a time when the user has eaten last time, and/or what was the 
user's activity 1 hour before the detection of the condition occurred, etc. The request for the 
media data may be created by virtue of Structured Query Language (SQL) used to 
communicate with a database which may be stored externally or internally at the external 
source or at the device respectively. It is known in the prior art that SQL statements are 
commonly used to perform tasks such as update data on a database, or retrieve data from a 
database. 

Clearly, the media data to be found may not be known to the search means 
before the suggestion is obtained. The media data may be found in the storage means of the 
device 100 further arranged to store the media data. Alternatively, the device 100 may 
comprise communication means (not shown) capable of downloading media data from an 
external source via a satellite, terrestrial or cable link, or cellular phone network link etc, 
wherein the search means is coupled to the communication means for retrieving the found 
media data. The media data may be retrieved from the Internet using publicly accessible 
Internet search engines such as "Google.com", "Altavista.com" etc, and generally from any 
apparatus storing the media data and connected to the device 100 via said communication 
means etc. In one example, the communication means may be a tuner/receiver arranged for 
receiving broadcast television or other signals, and/or a modem arranged for transmitting the 
request for the media data and/or for downloading the found media data. 

The device 100 may be incorporated into an apparatus such as a television set, 
video cassette recorder, personal digital assistant, personal computer, wearable equipment. In 
that way, the functions of such apparatuses can be extended with functions of the device 
according to the invention. 

A starting time of the suggested user's activity may be selected by the user, or 
by the device automatically with or without a user's confirmation. The starting time may 
trigger the presentation means to present the found media content to the user. The suggestion 
means may be arranged to enable the user to manually select the starting time for performing 
the suggested activity. In that case, the device may comprise input means, e.g. a keyboard, 
pointing means etc, arranged for providing a user's input to the suggestion means, whereas 
the presentation means may be arranged to facilitate the user input using the input means. 
Alternatively, the suggestion means may be arranged to generate a suggested starting time. 
For example, the suggested time may be generated on the basis of a standard time of doing 
the suggested activity in the detected conditions by an "archetype" of "regular user" stored in 
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the device, or the suggested means may analyze a history of the user's behavior if it was 
recorded in the device. The suggested starting time may be presented to the user by the 
presentation means 120, and the user may be enabled to confirm presenting the media content 
at the suggested starting time and/or to select another time using the input means. In turn, the 
5 suggestion means may allow the user to modify the starting time only within some time 
period or impose other limits on the user-inputted modifications. 

In one of the embodiments of the present invention, the presentation means 

may be further arranged to show for a predetermined time before said starting time a visual 
representation of a clock indicating at least said starting time. Being presented with such 
10 clock, the user may be reminded about some suggested activity in a user-friendly and 
unobtrusive way. The device may be arranged to show detailed information about the 
suggestion and/or the corresponding found media content upon a user's request. The user 
may realize such suggestion information request, for example, by using the pointing input 
device, e.g. mouse or trackball input device, to click on the visual representation of the clock 
1 5 shown on the presentation means. 

The visual representation of the clock may be standard for all suggestions. 
Alternatively, the detected conditions and/or the suggestions may be divided into different 
categories with respect to the types of user's activity, such as cooking a meal, physical 
training, cleaning a household etc. The visual representation maybe different for the different 
20 categories of the detected conditions and/or the given suggestions. 

The functions of the suggestion means, search means and other means, when 
appropriate, may be implemented using a microprocessor (not shown) coupled to random 
access memory and read-only memory (ROM) storing a program which when executed by 
said microprocessor can perform functions of said means as described above. The storage 
25 means may be realized with said ROM. The media data may be stored locally in the device or 
downloaded from the external source. The realization of such microprocessor system with 
corresponding internal and external circuits will be apparent to the person skilled in the art 
and need not be discussed herein. 

With reference to Fig. 2, an embodiment of the method of suggesting the user 
30 according to the present invention is shown. The method comprises a step 21 0 of detecting 
the condition pertaining to the user. The method may comprise a step 220 of automatically 
generating said suggestion to perform said activity in response to detection of said condition. 
The method further comprises a step 230 of automatically finding media data which provide 
the user with the direction, in other words which guide the user, for performing said 
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suggested user's activity, and a step 240 of presenting to the user said found media data 
which pertain to the suggestion. Other embodiments of the method, which correspond to the 
described above embodiments of the device of the present invention, may be envisaged. 

The various program products may implement the functions of the device and 
method of the present invention and may be combined in several ways with the hardware or 
located in different other devices. Variations and modifications of the described embodiment 
are possible within the scope of the inventive concept. Thus, for example, the use of the verb 
'to comprise' and its conjugations does not exclude the presence of elements or steps other 
lhan those defined in a claim. The invention can be implemented by means of hardware 
comprising several distinct elements, and by means of a suitably programmed computer. In 
the device claim enumerating several means, several of these means can be embodied by one 
and the same item of hardware. 

A 'computer program' is to be understood to mean any software product 
stored on a computer-readable medium, such as a floppy-disk, downloadable via a network, 
such as the Internet, or marketable in any other manner. 
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CLAIMS: 

09. 09.2002 



1. A data processing device for suggesting a user, comprising a detector arranged 

for detecting a condition pertaining to the user of said device, and presentation means for 
presenting to the user a suggestion to perform an activity in response to detection of said 
condition, characterized in that the device comprises 

search means arranged to automatically find media data which provide the 
user with a direction for performing said suggested user's activity, 

wherein said presentation means is arranged for presenting said found media 
data to the user. 



2 " The device °f claim 1 , wherein said detector is arranged for monitoring user's 

activities. 



3 - 11x6 device °f claim 1 or 2, further comprising suggestion means arranged for 
autonomously generating said suggestion to perform said activity in response to detection of 
said condition. 

4 - 1116 device of claim 1, wherein said device is arranged to enable the user to 
manually select and schedule a suggestion being a reminder to perform an activity. 

5. The device of claim 3, wherein said suggestion means is arranged for 
suggesting the user to start at least one physical exercise, said search means being arranged 
for finding media information guiding the user to perform at least one suggested physical 
exercise. 

6. The device of claim 3, wherein said suggestion means is arranged to suggest to 
the user a starting time for performing said suggested activity. 



7. The device of claim 3, wherein said suggestion means is arranged to enable 

the user to manually select a starting time for performing said suggested activity. 
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8. The device of claim 6 or 7, wherein said presentation means is arranged to 

show for a predetermined time before said starting time a visual representation of a clock 
indicating at least said starting time. 

9 < The device of claim 1 , wherein said search means is further arranged to obtain 
said-media-data from an external source. 

10. An apparatus incorporating the device as claimed in any one of the preceding 

claims, the apparatus being selected from a group of a television set, video cassette recorder, 
personal digital assistant, personal computer, wearable equipment. 

1 1 # A method of suggesting a user, comprising a step of detecting a condition 

pertaining to the user and a step of presenting to the user a suggestion to perform an activity 
in response to detection of said condition, characterized in that the method comprises: 

a step of automatically finding media data which provide the user with a 
direction for performing said suggested user's activity, and 

a step of presenting said found media data to the user. 

12. A computer program product enabling a programmable device when executing 

said computer program product to function as the device as defined in claim 1. 
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ABSTRACT: 



10 



The invention relates to a data processing device (100) for suggesting a user 
(101), comprising a detector (1 10) arranged for detecting a condition (1 15) pertaining to the 
user of said device, and presentation means (120) for presenting to the user a suggestion to 
perform an activity in response to detection of said condition. The device further comprises 
search means (140) arranged to automatically find media data (141) which provide me user 
with a direction for performing said suggested user's activity (131), wherein said presentation 
means is arranged for presenting said found media data to the user. The found media data, 
when are presented to the user, elucidate to the user how the suggestion can he realized. For 
instance, the device may show to the user a video clip demonstrating how suggested physical 
exercises may he performed. The invention also relates to a method of suggesting the user. 

Fig. 1 
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